Securing File Delivery with Terraform, Lambda@Edge, CloudFront, and S3
Introduction:
Delivering files securely while ensuring proper authentication is crucial for many applications. In this tutorial, we will explore how to leverage the power of Lambda@Edge, CloudFront, and S3 to create a robust file delivery system with authentication capabilities.
Prerequisites:
Before we begin, make sure you have the following:
1.An AWS account 2.Basic knowledge of AWS Lambda, CloudFront, and S3
Diagram
I tried
Step 1: Set up an S3 bucket: First, create an S3 bucket and upload the files you want to deliver securely. Ensure that the bucket permissions allow access from CloudFront.
Step 2: Configure CloudFront: Create a CloudFront distribution and configure it to use your S3 bucket as the origin. Specify the caching behavior, default root object, and any additional settings according to your requirements.
Step 3: Create an AWS Lambda function: Now, it's time to create the Lambda function that will handle authentication and request processing at the edge locations. You can write this function in the Lambda console or use your preferred programming language. Remember that the function runs in response to CloudFront events, so you can take advantage of the viewer-request event to perform authentication.
Step 4: Associate the Lambda function with CloudFront: Associate the Lambda function you created in the previous step with the appropriate CloudFront distribution. This ensures that the function is triggered for each incoming request at the edge locations.
Step 5: Implement authentication logic: Within your Lambda function, implement the authentication logic based on your requirements. This can involve checking credentials, validating tokens, or any other authentication mechanism you prefer. You can access request information and modify headers using the event object provided to the function.
import base64
# username = "admin"
# password = "secret123"
# credentials = f"{username}:{password}"
# base64_credentials = base64.b64encode(credentials.encode("utf-8")).decode("utf-8")
#Base64 encoded username:password
AUTHORIZATION = "Basic YWRtaW46c2VjcmV0MTIz"
ERROR_RESPONSE_AUTH = {
'status': '401',
'statusDescription': 'Unauthorized',
'body': 'Authentication Failed',
'headers': {
'www-authenticate': [
{
'key': 'WWW-Authenticate',
'value': 'Basic'
}
]
}
}
def lambda_handler(event, context):
request = event['Records'][0]['cf']['request']
headers = request['headers']
# Retrieve the Authorization header from the request
auth_header = headers.get('authorization', None)
if auth_header:
# Decode the Base64 encoded username:password
decoded_auth = auth_header[0]['value']
# Check if the decoded username:password matches the expected value
if decoded_auth == AUTHORIZATION:
return request
# If the authorization fails or no authorization header is present, return the authentication error response
return ERROR_RESPONSE_AUTH
Step 6: Grant access or return an error response: Based on the authentication result, you need to either grant access to the requested file or return an error response. If the authentication is successful, modify the request headers in the viewer-request event to allow access to the file. On the other hand, if authentication fails, return an appropriate error response to prevent unauthorized access.
Step 7: Deploy the Lambda function: Once you have implemented and tested your Lambda function locally, it's time to deploy it to the AWS Lambda service. Deploying the function makes it available for execution across the CloudFront edge locations.
Step 8: Test the authentication flow: Finally, test the authentication flow by accessing your files through the CloudFront distribution URL. Verify that the authentication logic is working as expected. Test both successful authentication and failed authentication scenarios to ensure the proper handling of access requests.
Architecture
Settings
the link provideded in this blog could be in japanese please translate if required
S3 Bucket general settings
The configuration includes two buckets: bucket1 and bucket2. Bucket1 contains all the files, while bucket2 is set up to redirect requests to bucket1.
| Configuration | bucket1 | Remark |
|---|---|---|
| Name | www.xxxx.info | To ensure uniqueness and alignment with static website hosting domains, the bucket name should match the hosted domain. |
| AWS Region | us-east-1 | the cost of us-east-1 is cheaper then other region for PerGB storage and easily configured to service like cloudfront and lambda@edge |
| Object Ownership | ACLs disabled | |
| Block Public Access settings for this bucket | Disable for website hosting | Enable this setting to restrict public access to the bucket and its objects. only rest api can be used after enabling |
| Bucket Versioning | Enable | Enabling versioning allows recovery from unintended user actions or application failures by preserving all versions of objects. |
| Tags | Name: .xxx.xxxxxx.xxx ManagedBy: Management Console Environment: Dev |
Tags can help with resource identification and management. |
| Encryption Type | SSE-S3 | Server-side encryption with SSE-S3 ensures data at rest is encrypted using AWS S3-managed keys. |
| Bucket Key | Enable | Enabling Bucket Keys reduces the request traffic from Amazon S3 to AWS KMS and reduces the cost of SSE-KMS. |
| Object Lock | Disabled | Object Lock provides an additional layer of protection by preventing object deletion or modification for a specified retention period. |
| Multi-factor authentication (MFA) delete | Disabled | Enabling MFA delete adds an extra layer of security, requiring MFA authentication for object and version deletion operations. |
Other S3 Settings
| Configuration | values | Remarks |
|---|---|---|
| Intelligent Tiering | N/A | https://dev.classmethod.jp/articles/amazon-s3-intelligent-tiering-further-automating-cost-savings-for-short-lived-and-small-objects/ |
| Server Access logging | Enable | |
| Target Bucket | s3://logs/s3 | |
| Lifecycle rules | N/A | https://dev.classmethod.jp/articles/understand-how-the-s3-lifecycle-rules-work/ |
| Replication rules | N/A | |
| Inventory configurations | N/A | https://dev.classmethod.jp/articles/s3-inventory-reinvent/ |
| Access Points | N/A | https://dev.classmethod.jp/articles/s3-access-restrict/ |
CloudFront Distribution
| Configuration | values | Remarks |
|---|---|---|
| Distribution domain name | dxxxxxxxx.cloudfront.net | |
| Origin domain | www.xxx.xxxxxx.xxx.s3.us-east-1.amazonaws.com | this time we are using s3 as origin |
| Origin path | - | |
| Name | www.xxx.xxxxxx.xxxinfo | |
| Origin access | public | to only allow access from cloudfront to s3 |
| Add custom header | - | |
| Enable Origin Shield | - | https://dev.classmethod.jp/articles/amazon-cloudfront-support-cache-layer-origin-shield/ |
| Connection attempts | 3 | |
| Connection timeout | 10 |
Behavior
| Configuration | Values |
|---|---|
| Behavior | Default cache behavior |
| Compress objects automatically | No |
| Viewer protocol policy | Redirect HTTP to HTTPS |
| Allowed HTTP methods | GET, HEAD |
| Restrict viewer access | No |
| Cache key and origin requests | Legacy cache settings Minimum TTL: 0 Maximum TTL: 31536000 Default TTL: 86400 sec |
| Response headers policy | - Read more |
| Smooth streaming | No |
| Enable real-time logs | No |
| Function association | Yes (refer to a different table) |
| Web Application Firewall (WAF) | Do not enable security protections |
Function association:
| event | function Type | Function ARN / Name ｜Include body |
|---|---|---|
| Viewer request | Lambda@edge | arn:aws:lambda:us-east-1:xxxxxxxxxx:IPAuth:x|no |
Settings
| Configuration | values | Remarks |
|---|---|---|
| Price class | all edge location | all edge location provide best performance |
| Alternate domain name (CNAME) | www.xxx.xxxxxxx.info | This is the domain name to be set on the CloudFront side. Access to CloudFront will be possible using this domain name. |
| Custom SSL certificate | www.xxx.xxxxxxx.info | If using ACM the certificate should be in us-east-1 region |
| Supported HTTP versions | HTTP 1.0, HTTP 1.1,HTTP/2 | |
| Default root object | index.html | https://dev.classmethod.jp/articles/cloudfront-distributions-should-have-a-default-root-object-configured/ |
| Standard logging | Off | |
| S3 bucket for logging | - | https://dev.classmethod.jp/articles/cloudfront-access-log-dont-choose-no-acl-s3-bucket/ |
| IPv6 | off | |
| Security Policy | TLSv1 |
Lambda@Edge
| Configuration | value | Remarks |
|---|---|---|
| Function name | dio-dev-lambda-function | |
| Runtime | Python 3.7 | |
| Architecture | x86_64 | graviton instance will give better price performance |
| Update runtime version | Auto | |
| Role name | cloudfront_access_lambda | |
| Policy name | my_inline_policy | |
| Enable Code signing | No check | |
| Enable function URL | No check | |
| tags | ManagedBy terraform Project dio-dev-blog | |
| Configure cross-origin resource sharing (CORS) | no check | |
| Enable VPC | No check | |
| Memory | 128MB | |
| Ephemeral storage | 512MB | |
| SnapStar | None | |
| Logs and metrics (default) | Enabled | |
| Enhanced monitoring | Not enabled | |
| Code profiling | Not enabled | |
| Active tracing | Not enabled |
Route 53
| Record name | Type | Routing policy | Alias | Route traffic to | TTL (seconds) | Evaluate target health |
|---|---|---|---|---|---|---|
| www..xxx.xxxxxx.xxxinfo | A | Simple | Yes | dxxxxxxxxxxx.cloudfront.net. | - | yes |
Template
data "aws_caller_identity" "current" {}
#redirect to other bucket
resource "aws_s3_bucket" "redirect_website" {
bucket = "xxx.xxxxxx.xxxinfo"
tags = {
Name = ".xxx.xxxxxx.xxxinfo"
Environment = "Dev"
}
}
resource "aws_s3_bucket_website_configuration" "redirect_website_configuration" {
bucket = aws_s3_bucket.redirect_website.bucket
redirect_all_requests_to{
host_name = aws_s3_bucket.static_website.bucket
}
}
#for static website hosting keep the bucketname same as domain name
resource "aws_s3_bucket" "static_website" {
bucket = "www.xxx.xxxxxx.xxx.info"
tags = {
Name = "www.xxx.xxxxxx.xxx.info"
Environment = "Dev"
}
}
resource "aws_s3_bucket_public_access_block" "public_access_block" {
bucket = aws_s3_bucket.static_website.id
ignore_public_acls = false
block_public_acls = false
restrict_public_buckets = false
block_public_policy = false
}
resource "aws_s3_bucket_policy" "allow_access_from_internet" {
bucket = aws_s3_bucket.static_website.id
policy = data.aws_iam_policy_document.allow_access_from_internet.json
}
data "aws_iam_policy_document" "allow_access_from_internet" {
statement {
principals {
type = "*"
identifiers = ["*"]
}
actions = [
"s3:GetObject"
]
resources = [
"${aws_s3_bucket.static_website.arn}/*",
]
}
}
resource "aws_s3_bucket_versioning" "versioning_example" {
bucket = aws_s3_bucket.static_website.id
versioning_configuration {
status = "Enabled"
}
}
resource "aws_s3_bucket" "log_bucket" {
bucket = "static-website-log-bucket"
}
resource "aws_s3_bucket_logging" "static_website_log" {
bucket = aws_s3_bucket.static_website.id
target_bucket = aws_s3_bucket.log_bucket.id
target_prefix = "log/"
}
#uploading the static website files to s3
# resource "aws_s3_object" "object" {
# for_each = fileset("build/", "**")
# bucket = aws_s3_bucket.static_website.id
# key = each.value
# source = "build/${each.value}"
# content_type = "text/html"
# }
resource "aws_s3_object" "object" {
bucket = aws_s3_bucket.static_website.id
key = "index.html"
source = "./index.html"
content_type = "text/html"
}
resource "aws_s3_object" "object2" {
bucket = aws_s3_bucket.static_website.id
key = "lockedindex.html"
source = "./lockedindex.html"
content_type = "text/html"
}
resource "aws_s3_bucket_website_configuration" "website_configuration" {
bucket = aws_s3_bucket.static_website.bucket
index_document {
suffix = "index.html"
}
}
#if want to use s3 without cloudfront
# resource "aws_route53_record" "route53_record1" {
# zone_id = "Zxxxxxxxxxxxxxxxx"
# name = "www.xxx.xxxxxx.xxx.info"
# type = "A"
# alias {
# name = aws_s3_bucket.static_website.website_domain
# zone_id = aws_s3_bucket.static_website.hosted_zone_id
# evaluate_target_health = true
# }
# }
resource "aws_route53_record" "route53_record1" {
zone_id = "Zxxxxxxxxxxxxxxx"
name = "www..xxx.xxxxxx.xxxinfo"
type = "A"
alias {
name = aws_cloudfront_distribution.static-www.domain_name
zone_id = "Z2FDTNDATAQYW2"
evaluate_target_health = true
}
}
resource "aws_acm_certificate" "cert" {
domain_name = "www.xxx.xxxxxx.xxxinfo"
validation_method = "DNS"
tags = {
Environment = "dev"
}
lifecycle {
create_before_destroy = true
}
}
resource "aws_cloudfront_distribution" "static-www" {
origin {
domain_name = aws_s3_bucket.static_website.bucket_regional_domain_name
origin_id = aws_s3_bucket.static_website.id
}
enabled = true
default_root_object = "index.html"
aliases = ["www.xxx.xxxxxx.xxx.info"]
default_cache_behavior {
allowed_methods = [ "GET", "HEAD" ]
cached_methods = [ "GET", "HEAD" ]
target_origin_id = aws_s3_bucket.static_website.id
forwarded_values {
query_string = false
cookies {
forward = "none"
}
}
lambda_function_association {
event_type = "viewer-request"
lambda_arn = "${aws_lambda_function.lambda.arn}:${aws_lambda_function.lambda.version}"
include_body = false
}
viewer_protocol_policy = "redirect-to-https"
min_ttl = 0
default_ttl = 3600
max_ttl = 86400
}
restrictions {
geo_restriction {
restriction_type = "whitelist"
locations = [ "JP" ]
}
}
viewer_certificate {
acm_certificate_arn = aws_acm_certificate.cert.arn
ssl_support_method = "sni-only"
}
}
#Lambda用IAMロールの信頼関係の定義
data aws_iam_policy_document assume_role {
statement {
effect = "Allow"
principals {
type = "Service"
identifiers = [
"lambda.amazonaws.com",
"edgelambda.amazonaws.com"
]
}
actions = ["sts:AssumeRole"]
}
}
#Lambda用IAMロールの作成
resource aws_iam_role iam_for_lambda {
name = "cloudfront_access_lambda"
assume_role_policy = data.aws_iam_policy_document.assume_role.json
inline_policy {
name = "my_inline_policy"
policy = jsonencode({
Version = "2012-10-17"
Statement = [
{
Action = [
"lambda:InvokeFunction",
"lambda:GetFunction",
"lambda:EnableReplication",
"cloudfront:UpdateDistribution"
]
Effect = "Allow"
Resource = "*"
},
]
})
}
}
data archive_file lambda {
type = "zip"
source_file = "lambda/lambda.py"
output_path = "lambda_handler.zip"
}
resource aws_lambda_function lambda {
filename = "lambda_handler.zip"
function_name = "IPAuth"
role = aws_iam_role.iam_for_lambda.arn
handler = "lambda.lambda_handler"
source_code_hash = data.archive_file.lambda.output_base64sha256
runtime = "python3.7"
}
Conclusion:
By following these steps, you have successfully created a secure file delivery system using Lambda@Edge, CloudFront, and S3. The Lambda function at the edge locations handles authentication, ensuring that only authorized users can access the files. This combination of services provides efficient and secure file delivery with fine-grained control over access.
Additional Considerations:
You can enhance the authentication logic by integrating with other authentication services like AWS Cognito, OAuth providers, or custom user databases. Implementing caching mechanisms within CloudFront can further optimize the delivery of files while maintaining security. Monitor and log authentication attempts to identify any suspicious activities and improve the overall security of your application.
